Field of the Invention
The present invention relates to an image processing technique for image data.
Description of the Related Art
In an image capturing apparatus such as a digital camera or a digital video camera, light from an object is guided onto a CCD or CMOS sensor serving as an image capturing device via an image capturing optical system formed from lenses and the like to form an image. The light that is to reach the image capturing device passes through the image capturing optical system and is therefore affected by the image capturing optical system. For this reason, the captured image blurs and lacks sharpness. There is known a method of recovering sharpness using an inverse filter that corrects the blur characteristic of the image capturing optical system. Japanese Patent Laid-Open No. 2010-258628 discloses a technique of controlling the inverse filter in accordance with a noise amount that increases due to the influence of the inverse filter.
Super resolution technologies for generating a high-resolution video from a low-resolution video have a problem of tradeoff between noise and sharpness. For example, a learning-based super resolution technology generates an image of high sharpness in general. However, the output image readily includes noise, and a faulty image is sometimes generated. On the other hand, a reconstruction-based super resolution technique can generate only an image of relatively low sharpness, though the output image includes no noise.
However, since the noise amount perceived by human vision is not linear, correction by the filter is insufficient or excessive in the technique described in Japanese Patent Laid-Open No. 2010-258628. How noise looks changes depending on the background image, that is, a visual masking effect is known. The technique described in Japanese Patent Laid-Open No. 2010-258628 cannot sufficiently recover a blur because the masking effect is produced in a high-frequency texture region.
In the super resolution technologies as well, the problem of tradeoff between sharpness and noise is not sufficiently solved.
The present invention has been made in consideration of the above problems, and provides an image processing method of performing image processing in consideration of tradeoff between sharpness and noise.